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Abstract. Educational videogames can be designed to provide instructional feedback responsive to specific actions. 
However, existing research indicates that students tend to ignore the feedback provided. That is, students often use ineffective 
help-seeking strategies. Research on the topic of help-seeking in learning environments have primarily focused on the role of 
cognitive factors, the nature of the help, or issues of timing and frequency. There is a noticeable gap in understanding how to 
motivate the use provided feedback. This study examined the relation between incentivizing the use of feedback and providing 
an explanation of the game’s scoring rules on math learning in a pre-algebra videogame. A randomized-control design was 
used, comparing learning outcomes of students who received the incentive with those who did not. Results indicated that 
students given the incentive to use feedback had significantly higher normalized change scores on math items ( d = .53), with 
stronger effects for students with low academic intrinsic motivation (of = .88 - 1.17). 


1.0 INTRODUCTION 

When event data are used to evaluate 
performance and have consequential 
meaning for the student, feedback is usually 
given. As such, one affordance of games 
for learning is that they can be designed to 
provide instructional feedback responsive to 
specific actions. Germane to the discussion 
of student use of feedback is the research 
on help seeking. The type of help that is 
often used in help-seeking studies is 
tailored hints and feedback, or supporting 
information accessible in general help 
menus. However, the research on help- 
seeking indicates that when students reach 
an impasse, they either use ineffective help- 
seeking strategies, or avoid seeking help 
altogether, as reviewed by [1], 

To understand the relationship between 
help seeking and learning, researchers 
have primarily studied the role of cognitive 
factors (e.g., self-efficacy), the nature of the 
provided help (e.g., context-specific versus 
generalized principles), timing and 
frequency (e.g., on demand or system- 
initiated), and how the process of help 
seeking can be taught explicitly. There is a 


noticeable gap in understanding how to 
motivate students to engage effectively in 
help seeking and use provided feedback. 

Moreover, this study is focused on learning 
games where game play is directly 
integrated and linked with its corresponding 
academic content. If game play is to require 
the use, demonstration, and evaluation of 
the application of knowledge or skills in a 
particular domain, game students may need 
to know the criteria underlying their 
progress or lack thereof. Communication of 
a game’s scoring rules can be leveraged to 
make assessment criteria more explicit and 
transparent. 

This paper will describe a study that 
examined the impact of both incentivizing 
the access of feedback and providing 
different degrees of explanation of the 
game’s scoring rules on math learning and 
performance in an educational videogame 
that was designed to teach students about 
fractions. 

1 . 1 The Importance of Feedback 
for Learning 
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Feedback related to progress in learning 
tasks can generally have a positive effect on 
learning [2] There are certain conditions that 
make it effective — specifically, feedback 
should support clear understanding of the 
learning goals or objectives (and the criteria 
that define good quality), a way for students' 
to relate own performance to that goal, and 
then a indicate the path to achieve the goal 
if there is any discrepancy between the goal 
state and their own performance [3]-[5], 

However, the mere presence of feedback, 
hints, or available help is insufficient for 
learning. Review of the research on the use 
of feedback in technology-based 
environments indicates that students rarely 
access voluntarily available feedback. This 
is problematic as the lack of compliance 
with the treatment greatly reduces any 
potential effect on the outcomes. For 
example, Nelson [6] compared different 
levels of feedback on student achievement 
in an immersive learning environment. 
Results from the study indicate that most 
students did not access the feedback. Also, 
between students who were provided 
extensive feedback and students who were 
given moderate feedback, there were no 
statistical differences on the frequency of 
accessing the hints. In a study that 
examined the effect of providing user- 
initiated feedback via a pedagogical agent, 
Van Eck and Dempsey [7] also reported low 
levels of student access to the feedback. 
The authors concluded that further research 
is essential to figure out how to promote its 
use. 

1 .2 Increasing the Use of 
Feedback in Technology-Based 
Learning Environments via 
Incentives 


The use of incentives for seeking help and 
feedback provides a relevant avenue for 
exploration. That students avoid help 
seeking is not surprising given that 
historically, it has been discouraged in most 
learning contexts. Research conducted in 
classroom settings suggests that there is 
social stigma attached to help seeking, a 
public sign of failure [8]-[10], Further, in 
game-based settings, accessing feedback 
slows the game down, and some games 
have speed of play as a basis of 
advancement. Moreover, in some 
technology-based environments, the access 
of feedback or available hints are 
associated with lesser proficiency or may be 
discouraged. For example, when a hint is 
given to a student in the PACT Geometry 
tutor, the student’s visible “skill bar” 
decreases [1 1], In games, students are 
often penalized when accessing help. In is 
discouraged (i.e., students are penalized 
when a hint is accessed). 

The design approach used in this study 
adopted the perspective that incentives can 
communicate what is valued within a given 
context and promote desired behavior. 
Rewards and incentives are, after all, a 
method for signaling those actions or 
behaviors that are encouraged or 
discouraged by a particular community. 
Conveying these expectations is a key 
process through which individuals learn how 
to participate in the communities that they 
are members of [1 2]-[1 3] . The use of an 
incentive to access feedback may be one 
potential approach, especially as findings 
from studies using tasks with initial low 
interest suggest that incentives may be 
beneficial [14]-[15], Moreover, in video 
games, incentives are intrinsically tied to 
performance and are often a permanent 
aspect of the activity. In fact, game 
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designers argue that incentives (either in 
the form of rewards or punishment) are 
essential to the fun or sometimes the 
compulsion of the experience [16]-[18], At 
best, an incentive can reverse the 
association between seeking help and 
failure, signaling to the student that seeking 
feedback is a valuable act leading to 
proficiency. 

1 .3 Rubrics of Scoring Rules to 
Promote Clarity of Expectations 

In formal learning settings, scoring rubrics 
(or codified scoring rules and corresponding 
score values) have been used to improve 
assessment clarity by making explicit what 
constitutes good performance. There have 
been numerous studies on the teacher use 
of rubrics, typically for student-constructed 
responses. Most of the research has 
examined how best to train teachers to 
effectively use rubrics to increase the 
reliability and validity of scoring 
performance assessments [19], 
Opportunities for student use of rubrics to 
improve learning appears logical, although 
only a few studies have examined this idea 
directly [20]-[22], 

When the scoring rules are appropriately 
tied to academic progress, providing an 
explanation of the game’s scoring rules 
functions as a rubric in a game for learning. 
The scoring rules make explicit the stated 
learning objectives of the game as well as 
the criteria used to evaluate performance. 
The scoring rules can direct attention to 
what and how responses are being scored 
which may (a) make more explicit the 
learning objectives or goals of the game, (b) 
clarify the criteria of performance (i.e., what 
“counts”), and (c) support the development 
of self-assessment of performance to 
determine when additional help is 


necessary. This information can be 
provided prior to game play to guide 
performance, or as a context for elaborated 
feedback. 

1.4 Summary 

The review of the literature indicates for 
learning environments to be effective, 
students need to understand what 
constitutes good performance and use 
provided feedback in order to circumvent 
the ineffectiveness of trial-and-error 
learning. The impact of two aspects of 
design features in game-based 
environments were examined on math 
achievement, game play, and use of 
feedback: (a) incentivizing the use of 
feedback, and (b) testing of different levels 
of making performance criteria explicit 
through the explanation of scoring rules. 

2.0 METHODOLOGY 

The following section will describe the 
research question addressed by the study, 
the research design, descriptions of the 
treatment conditions and dependent 
measures used in the study and the 
analyses of the data. 

2.1 Research Question 

In a game designed to teach the addition of 
rational numbers, are there differential 
effects on math achievement measures, 
game performance measures, and access 
of feedback help based on the amount of 
the use of an incentive to seek feedback 
and the amount of explanation of the 
scoring rules provided? 

2.2 Research Design 

Data was collected from 112 students in 
fourth to sixth grades in after-school 
contexts. A randomized-control, 1 * 4 
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design was used in this study. There were 
four treatment conditions, which are 
described in the following section. Within 
participating after-school programs, each 

Table 1 contains descriptions of the four 
treatment conditions used in the study with 
regard to the amount of scoring rules 
Table 1 . Description of Treatment Conditions 


2.3.1 Experimental Materials: Save 

Patch 

The game used in the study targeted two 
key mathematics ideas: (a) only identical 
units can be added to create a single 
numerical sum and (b) the size of a rational 
number is relative to how one whole unit is 
defined. 

The objective of the game was to help the 
game character (Patch) jump over obstacles 
(e.g., spikes, lava, quicksand) and move 
from block to block to reach the last “X” 
block (the final goal). To do this, students 


participant was randomly assigned to one of 
the four conditions. 

2.3 Treatment Conditions 

explanation that was provided and the 
incentive to use feedback. 

in the Study 


needed to compute the distance of the 
jump, place trampolines on the blocks, and 
add enough coils to the trampolines to make 
Patch bounce. The size of the coil 
determined how far Patch would bounce. 

For example, a one-half unit coil would 
cause Patch to jump over a one-half unit 
interval. 

The first part of the game required the 
student to determine the size of the intervals 
of the grid. The intersection of vertical red 
bars indicated the boundaries of the whole 
unit. The green dots broke up the whole 
unit into intervals (see Figure 1). 


Used 



Amount of information player was given regarding game’s 
scoring rules 


Treatment 

condition 

Explanation of 
scoring rules 
given prior to 
game play 

Scoring 
explanations 
available in 
general help 
menu 

Explicit 
feedback 
about when 
and how many 
points are 
earned and 
lost 

Math 

elaborated 
feedback of 
scored event 

After a mistake 
was made, player 
was given an 
incentive to 
access 
additional 
feedback help 

No scoring 
information 

O 

o 

o 

O 

o 

Points-only 

feedback 

O 

o 

X 

O 

o 

Explanation 
of scoring 
rules 

X 

X 

X 

X 

o 

Rewarding 
help seeking 

X 

X 

X 

X 

X 
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Figure 1. The intersection of the vertical bars depicted the boundaries of the 
whole unit. The green dots broke up the whole unit into intervals. 


Once the students have figured out the unit 
size of the spaces on the grid, they needed 
to add together the correct number of coils 
that will span the distance to jump over. For 
example, in Figure 3, to get safely from one 
block to the next, Patch needs to jump over 
three one-third-unit intervals. This means 
that to successfully make it to the next 
block, the trampoline must contain three 
one-third-unit-sized coils. If the trampolines 
did not contain the correct number or size of 


coils necessary to get Patch to the next 
block, Patch exploded into feathers and the 
students were allowed to replay the level. 
To reinforce the idea that only fractions with 
like denominators can be added together, 
the student could not combine coils that 
have different unit sizes. If a student had a 
trampoline with a one-third coil on it and 
tried to add a one-sixth coil, the one-sixth 
coil would not go onto the trampoline. 
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Figure 2. Jump distance equals total number of spaces between blocks. The 
number of coils to add to a trampoline equals the jump distance. 


2.3.2 Save Patch Instructional 
Features 

Tutorials : The design of the tutorial was 
meant to first contextualize the math 
concept within the game, and then discuss 
how that concept relates to math in general. 
The tutorial information was presented as 
both written text and guided interaction. 

Scored events during game play. Across all 
of the conditions, three events were chosen 
as key points where performance would be 
evaluated because they mapped onto the 
learning objectives of the game. Points 
were earned any time the following event 
occurred: (a) student used coils that were 
the correct unit size for the grid, (b) student 
added together coils with like denominators, 
and (c) student successfully completed the 
level. Points were lost when any of the 
following events occurred: (a) student used 
coils that were not the correct unit size for 
the grid, (b) student attempted to add coils 
with unlike denominators together, and (c) 


student failed to get Patch to an 
intermediate goal. 

Feedback: All of the players in each 
condition received feedback after three 
events, that was in the form of the 
knowledge of results without elaboration or 
explanation. When a player tried to add a 
coil to a trampoline that had a coil with 
unlike denominators, the coils would not 
combine on the trampoline. Also, if the 
trampoline did not contain the correct-sized 
coil, or amount necessary to cover the 
distance of the jump, Patch exploded into 
feathers. When Patch jumped from block to 
block without exploding, this indicated that 
the player placed the correct quantity of 
coils needed to make the jump. 

General help menu : All of the players in 
each condition had access to a general help 
menu at any time throughout the game 
levels. The topics that were included in the 
general help menu provided information on 
game mechanics (e.g., how to make Patch 
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move in different directions) as well as 
instructional information such as how to 
choose the right-sized coil and how to add 
coils of different sizes. 

Feedback help. After the second 
consecutive mistake, all of the players in 

2.4 Dependent Measures 


each condition were given an opportunity to 
access additional feedback by clicking on a 
button that read, “Click here for help.” The 
additional feedback provided elaborated 
explanations and hints that were designed 
to assist the player with repairing the 
mistake. 


Table 3. Description of dependent measures used in the study. 


Math achievement 

Game performance 

Total proportion of times additional 
feedback was accessed 

Pretest: 31 items 

Number of: 

- Overall in the game 

Posttest: 44 items 

- coils added together 

- After coils with different denominators were 

(includes items in 

- wrong-sized unit coils used 

added together 

game context) 

- resets 

- When a wrong-sized unit coil was used 


- failed attempts 

- Failed attempt 


2.5 Analyses 

Three sets of orthogonal planned 
comparisons were conducted to examine if 
the data supported the hypotheses of the 
study. 

3.0 RESULTS 

Results indicated that compared to students 
who received minimal scoring information, 
students who were provided both 
explanation of the scoring rules and an 
incentive to seek additional feedback had 
significantly higher normalized gain scores 
on the math assessments (d= .53) and 
better game performance (d= .99 - 1 .09), 
with stronger effects for students with low 
academic intrinsic motivation (d= .88 - 
1.17). Students given the incentive 
accessed the feedback less frequently and 
spent the least amount of time on the 
feedback screens than the other students. 
However, when feedback was accessed, 
compared to the other students, they would 
solve the game level more quickly and with 
fewer mistakes. Furthermore, upon the 


occurrence of the same mistake, unlike the 
students in the other condition, those given 
the incentive would seek additional 
information in the help menu on the same 
topic. 

4.0 CONCLUSIONS 

Overall, findings suggest that when 
designing games for learning, it is important 
both to make performance criteria (in this 
case, the scoring rules of a game) explicit to 
a student and to find ways to motivate 
students to use provided feedback through 
the use of incentives. There are two 
potential explanations of the positive effect 
of the combined incentive and explanation 
of scoring rules. First, providing the 
rationale for the scoring rules may clarify the 
criteria of performance, (i.e., what “counts”). 
Second, the incentive signals that the use of 
feedback is valuable, increasing the 
likelihood that students will engage in self- 
directed help-seeking and deeper 
engagement with the information on the 
screen. Findings from this study suggest 
that it is important both to make assessment 
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criteria explicit to students and to find ways 
to motivate students to use provided 
feedback through the use of incentives. 
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